Discrete simulation optimization for tuning machine learning method hyperparameters

An important aspect of machine learning (ML) involves controlling the process for ML method in question to maximize its performance. Hyperparameter tuning (HPT) selecting suitable parameters that control process. Given HPT can be conceptualized as a black box optimization problem subject stochasticity, simulation (SO) methods appear well suited this purpose. Therefore, we conceptualize discrete SO and demonstrate use Kim Nelson (KN) ranking selection method, stochastic ruler (SR) adaptive hyperbox (AH) random search HPT. We also construct theoretical basis applying KN method. application SR wide variety models, including deep neural network models. then successfully benchmark KN, AH against multiple state-of-the-art methods.